WHAT IS CLAIMED IS : 

1 1. A method for matching a two dimensional image to one of 

2 a plurality of three dimensional candidate models, the method 

3 comprising the steps of: 
4 

5 determining the position and orientation of the two 

6 dimensional image; 
7 

8 for each three dimensional candidate model, computing a 

9 histogram-like table having a computed brightness coefficient for 

10 each surface normal of the three dimensional candidate model 

11 based on the corresponding value in the two dimensional image, 
1§] each brightness coefficient depending only on the corresponding 
l;P surface normal; 

l5 successively rendering each three dimensional candidate 

model in the determined position and orientation using the 

ljO surface normals in conjunction with the corresponding computed 

% brightness histogram-like table; and 

m 

ti£ comparing the two dimensional image with each of the 

25 rendered three dimensional candidate models. 

1 2. The method of claim 1, further comprising the steps of 

2 determining if the two dimensional image matches one of the 

3 plurality of three dimensional models if the comparison between 

4 the two dimensional image and the rendered three dimensional 

5 model is within a predetermined allowable error. 

1 3. The method of claim 1, wherein the comparison step 

2 results in a ranking for each of the rendered three dimensional 

3 models which indicates the likelihood that the corresponding 

4 three dimensional model matches the two dimensional image. 
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1 4. The method of claim 3, further comprising the step of 

2 choosing the three dimensional model corresponding to the 

3 rendered three dimensional model having the highest ranking as 

4 the three dimensional model which matches the two dimensional 

5 image. 

1 5. The method of claim 1, further comprising the step of 

2 reading the three dimensional models from a database containing 

3 the plurality of three dimensional models. 

1 6. The method of claim 1, wherein the comparing step is 

2 done simultaneously with the rendering step for each rendered 
J three dimensional model. 

U] 7. The method of claim 1, wherein the computing step sets 

% a minimum threshold of reflectance to be used in the computation 

ITS of the histogram-like table. 

; — : 

l M 

3 1 8. The method of claim 1, wherein the computing step sets 

! =4 a minimum and/or maximum threshold of intensity to be used in the 
computation of the histogram-like table. 

iji 9. The method of claim 1, wherein the two dimensional 

2 image is a color image and wherein the a brightness coefficient 

3 is computed for each color component of the color image. 

1 10. The method of claim 1, further comprising the step of 

2 smoothing the lightsphere histogram-like table to improve the 

3 rendering of each three dimensional model and the performance of 

4 the method. 

1 11. The method of claim 10, wherein the smoothing step is 

2 done by a convolution with a small box function or other small 
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3 smoothing kernel . 

1 12 . A method for matching a two dimensional image to one of 

2 a plurality of three dimensional candidate models, the method 

3 comprising the steps of: 
4 

5 determining the position and orientation of the two 

6 dimensional image; 
7 

8 for each three dimensional candidate model, computing a 

9 histogram-like table having a computed brightness coefficient for 

10 each surface normal of the three dimensional candidate model 

11 based on the corresponding value in the two dimensional image, 
O each brightness coefficient depending only on the corresponding 
'M surface normal; 

;g computing the variance of the brightness coefficients 

|f that are used to create each bucket of the histogram-like table, 

l & a bucket being a set of similar normal values that are lumped 

together to form a single argument value in the histogram- like 

m table; 

: — : 

;a computing the sum of the variances of the histogram- 

! 22 like table buckets; and 

23 

24 ranking the three dimensional candidate models using 

25 its computed sum as an error function, the ranking indicating the 

26 likelihood that the corresponding three dimensional model matches 

27 the two dimensional image. 

1 13. The method of claim 12, further comprising the step of 

2 choosing the three dimensional model corresponding to the three 

3 dimensional model having the highest ranking as the three 

4 dimensional model which matches the two dimensional image . 
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1 14. The method of claim 12, further comprising the step of 

2 reading the three dimensional models from a database containing 

3 the plurality of three dimensional models. 

1 15. The method of claim 12, wherein the computing step sets 

2 a minimum threshold of reflectance to be used in the computation 

3 of the histogram-like table. 

1 16. The method of claim 12, wherein the computing step sets 

2 a minimum and/or maximum threshold of intensity to be used in the 

3 computation of the histogram- like table. 

J 17. The method of claim 12, wherein the two dimensional 

J} image is a color image and wherein the a brightness coefficient 

Ml is computed for each color component of the color image. 

rd 18. The method of claim 12, wherein the sum of the 

variances are weighted by the number of samples in each bucket. 

"zia? 

H 19 . A method for matching a two dimensional image to one of 

a plurality of three dimensional candidate models, the method 

Q comprising the steps of: 

: — : 
: : 

5 determining the position and orientation of an object 

6 giving rise to the two dimensional image; 
7 

8 computing a representation of lighting effects that 

9 allows the lighting that gave rise to the two dimensional image 

10 to be used to render a realistic image of a three dimensional 

11 candidate model; 
12 

13 successively rendering each three dimensional candidate 

14 model in the determined position and orientation using the 
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15 surface normals in conjunction with the corresponding computed 

16 representation of lighting effects; and 
17 

18 comparing the two dimensional image with each of the 

19 rendered three dimensional candidate models. 

1 20. A method for matching a two dimensional image to one of 

2 a plurality of three dimensional candidate models, the method 

3 comprising the steps of: 
4 

5 determining the position and orientation of an object 

6 giving rise to the two dimensional image; 
7 

4 computing a representation of lighting effects based on 
:j the plurality of three dimensional candidate models and the two 

Wj dimensional image that allows evaluation of the likelihood that a 

|§ particular three dimensional candidate model gave rise to a 

M particular two dimensional image; and 

s 

14 choosing the most likely three dimensional candidate 

:t$ model to have generated the query based on the computed 
representation of lighting effects. 

: — : 

JSf 21. A computer program product embodied in a computer- 

2 readable med\um for matching a two dimensional image to one of a 

3 plurality of Haree dimensional candidate models stored in a 

4 database, the computer program product comprising: 

6 computer \eadabie program code means for determining 

7 the position and orientation of the two dimensional image; 

9 computer /readable program code means for computing a 

10 histogram-like tal/le eachNthree dimensional candidate model, 

11. having a computed brightnes\ coefficient for each surface normal 
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12 of the three dimensional candidate model based on the 

13 corresponding value in the two dimensional image, each brightness 

14 coefficient depending only on the corresponding surface normal; 

15 \\ k 

16 \ / computer readable program code means for successively 

17 rendering^efech three dimensional candidate model in the 

18 determijne^ position and orientation using the surface normals in 

19 conjunction with the corresponding computed brightness histogram- 

20 like Arable ;\ and 

21 \ 

22 computer readable program code means for comparing the 

23 two dimensiona\ image with each of the rendered three dimensional 

24 candidate models^ 

:J! 2^. A program storage device readable by machine, tangibly 

embodying a program^ of instructions executable by the machine to 

r3j perform method steps 1 ±to$ matching a two dimensional image to one 

j 4j of a plurality of thr/ee. dimensional candidate models, the method 

jj comprising the steps o^: . 

'7% determining th^ position and orientation of the two 

\M dimensional image; 

2 

jH§ for each three dimensional candidate model, computing a 

11 histogram-like table having d computed brightness coefficient for 

12 each surface normal of the thitee dimensional candidate model 

13 based on the corresponding val\jV=i in the two dimensional image, 

14 each brightness coefficient depending only on the corresponding 
15. surface normal ; 

16 

17 successively rendering eadh three dimensional candidate 

18 model in the determined position and orientation using the 

19 surface normals in conjunction with ttfe corresponding computed 

20 brightness histogram-like table; and 
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21 comparing the two dimensional image with each of the 

22 rendered three dimensional candidate models. 

1 23. \ A computer program product embodied in a computer- 

2 readable medium for matching a two dimensional image to one of a 

3 plurality or\three dimensional candidate models stored in a 

4 database, the\omputer program product comprising: 

6 computet readable program code means for determining 

7 the position and orientation of the two dimensional image; 

8 \ 

9 computer readable program code means for computing a 

10 histogram-like table ror each three dimensional candidate model, 
XL having a computed brightness coefficient for each surface normal 
1=£ of the three dimensional\ candidate model based on the 

ii\ corresponding value in the two /dimensional image, each brightness 

M coefficient depending only\on yutie corresponding surface normal; 

: iJ \ / I J' 

\ / \ / 

; =? \/ \ S 

IS computer readable ©ro^am code means for computing the 

^ variance of the brightness caef f ^cients that are used to create 

m each bucket of the histogr^m-luke Yable, a bucket being a set of 

W similar normal values thait are \L™ped together to form a single 

0 \ 
argument value in the histogram Alike table; 

2B \ 

.22 computer readable program code means for computing the 

23 sum of the variances of the histogram- like table buckets; and 

24 \ 

25 computer readable program code means for ranking the 

26 three dimensional candidate models usang its computed sum as an 

27 error function, the ranking indicating Vthe likelihood that the 

28 corresponding three dimensional model matches the two dimensional 

29 image . \ 

nffr > 

1 . A program storage device readable by machine, tangibly 
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embodying a program of instructions executable by the machine to 
perform method steps for matching a two dimensional image to one 
of a plurality of three dimensional candidate models, the method 
comprising the steps of: 

determining the position and orientation of the two 
dimensional image; 

for each three dimensional candidate model, computing a 
histogram-like table having a computed brightness coefficient for 
each surface normal of the three dimensional candidate model 
based on the corresponding value in the two dimensional image, 
each brightness coefficient depending only on the corresponding 
surface normal; 

computing the variance of the brightness coefficients 
that are used to create each bucket of the histogram-like table, 
a bucket being a set of similar normal values that are lumped 
together to form a single argument value in the histogram- like 
table; 

computing the sum of the variances of the histogram- 
like table buckets; and 

ranking the three dimensional models using its computed 
sum as an error function, the ranking indicating the likelihood 
that the corresponding three dimensional candidate model matches 
the two dimensional image. 

25. A computer program product embodied in a computer- 

readable medium for matcMngV^two dimensional image to one of a 
plurality of three dimensional candidate models stored in a 
database, the computer program product comprising: 
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6 computer readable program code means for determining 

7 the position and orientation of an object giving rise to the two 

8 dimensional image; 
9 

10 computer readable program code means for computing a 

11 representation of\lighting effects that allows the lighting that 

12 gave rise to the two dimensional image to be used to render a 

13 realistic image of V thpee dimensional candidate model; 
14 

15 computer readable program code means for successively 

16 rendering each three dimensional candidate model in the 

17 determined position ^JL*6Mentation using the surface normals in 

18 conjunction with thgrcoYreaponding computed representation of 

19 lighting effects; 

I 

2Ti computer readable program code means for comparing the 

2Z two dimensional image with\each of the rendered three dimensional 

2|| candidate models. 

A program storage device readable by machine, tangibly 

'5 embodying a program of instructions executable by the machine to 

f£ perform method steps for matching a two dimensional image to one 

ii of a plurality of three dimensional candidate models, the method 

y comprising the steps of: 
6 

7 determining the position and orientation of an object 

8 giving rise to the two dimensional image; 
9 

10 computing a representation of lighting effects that 

11 allows the lighting that gave rise to the two dimensional image 

12 to be used to render a realistic image of a three dimensional 

13 candidate model; 
14 

15 successively rendering each three dimensional candidate 



49 



16 model in the determined position and orientation using the 

17 surface normals in conjunction with the corresponding computed 

18 representation of lighting effects; and 
19 

20 comparing the two dimensional image with each of the 

21 rendered three dimensional candidate models. 

1 27. A computer program product embodied in a computer- 

2 readable medium for matching a two dimensional image to one of a 

3 plurality on three dimensional candidate models stored in a 

4 database, they computer program product comprising: 

6 computer readable program code means for determining 

_7 the position arm orientation of an object giving rise to the two 

^ dimens i onal imaae ; 
IJ5 \ 

$ computer reajdable program code means for computing a 

fflj representation of lighting effects based on the plurality of 

'3 three dimensional candidate models and the two dimensional image 

: 13 that allows evaluation \of^ the likelihood that a particular three 

P dimensional candidat^rlnbdel gave rise to a particular two 

M dimensional image ;^ancK \ 

O / \\ 

;g computer readable program code means for choosing the 

18 most likely three dimens ronal candidate model to have generated 

19 the query based on the computed representation of lighting 

20 effects. \ 

1 ,e 5^ A program storage device readable by machine, tangibly 

2 embodying a program of instructions executable by the machine to 

3 perform method steps for matching a two dimensional image to one 

4 of a plurality of three dimensional candidate models, the method 

5 comprising the steps of: 
6 
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7 determining the position and orientation of an object 

8 giving rise to the two dimensional image; 
9 

10 computer readable program code means for computing a 

11 representation of lighting effects based on the plurality of 

12 three dimensional candidate models and the two dimensional image 

13 that allows evaluation of the likelihood that a particular three 

14 dimensional candidate model gave rise to a particular two 

15 dimensional image; and 
16 

17 computer readable program code means for choosing the 

18 most likely three dimensional candidate model to have generated 

19 the query based on the computed representation of lighting 

20 effects. 

□ 
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